Infinite-dimensional gradient-based descent for alpha-divergence minimisation

This paper introduces the (?,?)-descent, an iterative algorithm which operates on measures and performs ?-divergence minimisation in a Bayesian framework. gradient-based procedure extends commonly-used variational approximation by adding prior parameters form of measure. We prove that for rich family functions ?, this leads at each step to systematic decrease derive convergence results. Our framework recovers Entropic Mirror Descent provides alternative we call Power Descent. Moreover, its stochastic formulation, (?,?)-descent allows optimise mixture weights any given model without information underlying distribution parameters. renders our method compatible with many choices updates applicable wide range Machine Learning tasks. demonstrate empirically both toy real-world examples benefit using going beyond framework, fails as dimension grows.